1. Field of the Invention
The present invention generally relates to computing systems, and specifically to systems that use packet-switching fabrics, such as the InfiniBand architecture.
2. Discussion of the Related Art
In current-generation computers, the central processing unit (CPU) is connected to the system memory and to peripheral devices by a shared parallel bus, such as the Peripheral Component Interface (PCI) bus or the Industry Standard Architecture (ISA) bus. Essentially, a bus is the channel or path between components in a computer. Likewise, current server to server connections and links to other server-related systems, such as remote storage and networking devices, depend on parallel bus technology. Server design dependent on a shared bus input/output (I/O) architecture may deliver for example 512 MB/sec of potential bandwidth that is shared among devices connected to the bus. As additional connections are added to the server, potential performance-per-connection decreases and I/O contentions escalate.
As data path-widths grow, and clock speeds become faster, the shared parallel bus becomes too costly and complex to keep up with system demands. In response, the computer industry is working to develop a next-generation bus standard. Many of the proposed new standards have something in common. They propose doing away with the shared-bus technology used in PCI and moving to a point-to-point switching connection. Thus, the computer industry is moving towards fast, packetized, serial input/output bus architectures, in which computing hosts and peripherals are linked by a switching network, commonly referred to as a switching fabric. A number of architectures of this type have been proposed, and the first next-generation standard is already in place. InfiniBand has been advanced by a consortium headed by a group of industry leaders.
The InfiniBand architecture is an I/O infrastructure technology that simplifies and speeds server-to-server connections and links to other server related systems, such as remote storage and network devices. The Infiniband fabric is the central network of connections between servers and remote networking and storage devices within a data center. Infiniband architecture also constructs highly available data centers through multiple levels of redundancy. By connecting nodes via multiple links, Infiniband systems continue to perform even if one link fails. For increased reliability, multiple switches in a fabric provide redundant paths that allow for seamless re-routing of data through the fabric should a link between switches fail. Completely redundant fabrics may be configured for the highest level of reliability and may continue to perform even if an entire fabric fails.
A common problem for any network is how to provide Quality of Service (QoS) through a network. To provide QoS, the network traffic must be differentiated. Some traffic should be handled in one way, other traffic another.
For example, an entity may set up a Service Level Agreement (SLA) with a network service provider (the entity that is providing the network), that specifies that the entity's traffic will always have available a certain bandwidth (e.g., 10 Megabits per second, or 10 Mbps) and latency (e.g., less than 1 millisecond, or ms). Then, whenever a packet is detected that comes from or is going to that entity, the packet should receive special handling. If the overall flow to the entity is currently less than 10 Mbps, then the packet should get through without being dropped and with a latency of less than 1 ms. This type of flow is said to be handled by Assured Forwarding (AF). Packets arriving when the current flows are greater than 10 Mbps will be handled differently, perhaps as Best Effort (BE) traffic (see below).
As another example, a router could be set up to recognize certain types of flows as real-time flows. Real-time flows are characterized by the idea that if the packet doesn't arrive in time it might as well not arrive at all. For example, a packet of voice data in a telephone conversation has to be available at the receiver when it is needed, or it is useless. Too late, and it cannot be used and will just be discarded. So real-time traffic (a stream of packets) belonging to a voice conversation should be handled by a class of behavior known as Expedited Forwarding (EF). A packet handled this way will be forwarded very quickly (with low latency). Hopefully, the variation in latency (known as jitter) will also be low. As a tradeoff, packets in such a stream may be simply dropped if their aggregate bandwidth exceeds a certain threshold. Also, a SLA covering such packets may be expensive to the buyer because providing this kind of service requires that a router have features that make it expensive to build.
A third example is for traffic that is not covered by any SLA, which is called Best Effort (BE) traffic. This type of traffic is typically found now across the Internet. Best effort packets may be dropped for any reason, and have no particular guarantees about latency or jitter.
Therefore, one of the side effects of using multi-switch architectures is the need to decide which packets go on a switch, because any switch can get to any destination. Many possible algorithms may be implemented to aggregate data across multiple InfiniBand switch sub-networks. Accordingly, there is a need for an algorithm that is simple, fast, fair, and robust.